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Abstract 


The recent surge in industrial expansion can be attributed significantly to the advancements in automation 
technology. For better performance of industrial processes, automated systems are used. Image processing 
has played a great role in the applications of robotics and embedded systems. Sorting of objects is usually 
done by humans which takes a lot of time and effort. By employing image processing methods alongside 
appropriate sensors, object detection becomes feasible, enabling the utilization of robotic arms for sorting 
diverse items. This reduces human effort and improves the time to market the product. The model includes the 
conveyor system, capturing and detecting the objects and placing the objects. Using image processing 
techniques, the captured image is compared with its texture and color. Based on image processing, the robotic 
arm will be controlled and place the objects in desired locations. An Arduino model is used to process the 
captured image. The captured image is other than a pre-specified object, it involves the conveyor belt starting 
to move and the object is collected at the end of the conveyor system. 
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1. Introduction 

Significant advances have been made in robotics in 
the last two years. In the coming years, they will 
appear in space research and many activities in daily 
life. A robot is a mechanical device that performs 
tasks and acts according to a predefined set of general 
instructions and human control. Table 1 shows 
Literature Survey Analysis. These activities replace 
or support human activities such as the manufacture, 
operation, or transportation of heavy or hazardous 
substances. Robots are an important electronic and 
flexible product in high demand today. Robots are 
now more than machines, as labor costs and customer 
demand increase, robots are becoming the solution of 
the future. Robotic arms are used only for industrial 
automation and working in hazardous environments. 
Many robot controllers are expensive due to the 
special machining of high-precision drivers and 
components. In this report, a smart robotic arm 
received a command from a driver to pick up a 
specific object and then place it first. Figure 1 shows Figure 1 Robotic Arm Setup 
The Robotic Arm Setup [1]. 
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Table 1 Literature Survey Analysis 
Year 
Title Authors Of Work Done Results 
Paper 
This Project focuses on a 
Implementation of deep Chakraborty mechanism based on the available Detected the deep characteristics of 
learning methods to 2021 deep learning model to determine | fruits using a stereo camera and an 
identify rotten fruits.[2] Savon the fruit fast and reliably in a | indoor fruit dataset, resulting in efficient 
complicated orchard environment. identification of varied 
fruit sizes. 
In this project, a fruit classification 
Miah Md system was made useful in a variety _ 
Sohel of areas, including autonomous | By finding defects in agricultural fruits, 
agricultural robots and the | the expense and time for processing 
An advanced method of production of smartphone apps for | may be minimized. In the proposed 
Identification of fresh ang Tayeeba identifying unique fruit species on | classificati hey h ked 
: : : Tasnuva classification system, they have worke 
ieee nance 2021 | the market. In this study, they took a | on five CNN models. The Inception V3 
o Mirajul total of 5658 fruit images which are | model has an effective of 97.34 %. 
Networks [4] ne based on 10 classes. Using CNN, we 
Islam : ae : i 
can pick up design in the image, and 
is robust for computer vision. 
Ganesh 
Khekare 
Ajinkya The Authors have published That 
Shende automaon ls the agricultural sector | The project is realized to sort and pack 
increases and improves the different types of fruits. The objectives 
. standard, basic features, expansion : ane e : 
Aldrick ffici f th : ¢ | are also achieved which include: sorting 
Design of fruit segregatiq Gonsalves end Seieney oF he mmcducnone five different types of fruits, packaging 
and packaging machine.[ 2020 goods. The features Used in this fj € frui d "4 hi q 
were preprocessing, thresholding, a YP poe HS ee Ne eoane res 
Kaustubh segmentation, extraction, labelling the fruits. 
Padalkar classification and detection. 
Vinit 
Rodrigues 
This study comparatively analyses | The classification process is carried out 
Karakaya an image dataset containing samples through well-known support vector 
Diclehan of three types of fruits to distinguish | Machines-based classifiers. After 
AC eae fresh samples from those of rotten. | testing several experimental scenarios 
Of oe saa Oguzhan The proposed vision-based the including ; binary and multi-class 
Ciassifieation.(61 Ulucan 2019 framework utilizes histograms, classification problems, it turns out to 
. i gray-level co-occurrence matrices, have the highest success rates are 
Mehmet bag of features, and convolutional | Obtained consistently with the adoption 
Tiirkan. neural networks for feature | Of convolutional neural networks-based 
extraction. features. 
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2. Methodology 

To create our data, we have chosen the fruit as 
“apples”. We have segregated them based on their 
redness (Color texture). After that, we compared the 
apples based on their color. We processed most 
apples into each group (ripe and unripe) and then put 
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them into separate containers. The software part is 
implemented using [2] Arduino programming for 
controlling the hardware and Python programming 
for computer vision tasks. Figure 2 shows Block 
Diagram. 


USB WEB CAM 
(Mounted on 
Conveyer Belt) 


Servo Mechanism 


Servo Driver 


Servo Power Supply 


15W Adapter 


Conveyer Motor 


Motor Driver 


Any PC Supporting Windows OS 
Includes Python (with necessary lib) 


Power Supply 


Figure 2 Block Diagram 


Servo drives are electronic devices used to control 
servo motors. The servo drive receives feedback 
signals from the controller to indicate the desired 
position and speed. We use an arm-mounted RGB 
camera. The processor drives the 5 servo motors that 
make up the robot arm, and the camera is mounted on 
a bracket separate from the board. There are 2 boxes 
in the middle for storing [3] ripe and unripe fruits 
respectively. L298N is a motor controller that can 
control the speed and direction of two DC motors 
simultaneously. This module can drive DC motors. It 
is used to control two motor drivers, one for the 
conveyor belt and the other for the robot arm. We 
developed an application for a commercial robotic 
arm to sort, pick, and place fruits on a conveyor belt 
based on their physical state i.e. ripe vs. unripe. 
Figure 3 shows Circuit Diagram. This project 
involves the development of a system that can 
automatically sort apples based on their ripeness 
using computer vision and robotics. 


Figure 3 Circuit Diagram 


The system consists of hardware [4] components like 
an Arduino Nano microcontroller, a stepper motor 
with a conveyor belt, a robotic arm for picking and 
placing apples, and a USB camera connected to a 
Windows system. Figure 4 shows Actual Model 
Setup. 
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Workflow Model: 

Apple Detection: The Python script continuously 
analyzes the video feed from the USB camera to 
detect apples [5]. 

Ripeness Detection: After detecting an apple, the 
script calculates the percentage of red and yellow- 
green pixels in the apple's region of interest (ROI) to 
determine its ripeness. Figure 5 shows Workflow 
Model. 


Apple 
Detection 


Feedback 
Loan 


Robotic Arm 
Contirel 


Ripencss 
Detection 


omnia od 
Sending 


Figure 5 Workflow Model 


Command Sending: If a ripe or unripe apple is 
detected, the Python script sends a corresponding 
command (1 for ripe, 2 for unripe) to the Arduino 
Nano via serial communication. 

Robotic Arm Control: Upon receiving the 
command, the Arduino Nano controls the robotic arm 
to pick up the apple from the conveyor belt and place 
it in the appropriate bin. 

Feedback Loop: The Arduino Nano sends feedback 
(e.g.,  "Finished") to the Python script after 
completing the sorting task for each apple. This 
feedback ensures synchronization between the 
computer and the hardware components. Figure 6 &7 
shows the Detected fruits. 


Figure 6 Detection of Fruit 
The working of our project is: 

1. First, we keep the apple in the initial position of 
the conveyor belt where the camera detects the 
apple. 

2. The camera mounted on the start of the conveyor 
belt then detects the apple. 
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3. Using the Windows algorithm the apple texture 
is detected. 

4. The conveyor belt then starts to rotate 1.e. it takes 
the calculated steps towards the arm. 

5. Once it reaches below the arm gripper the arm 
starts picking up the apple. 

6. As per detected the arm places it in the desired 
container. 


Figure 7 Apple Detected 
3. Implementation 
Algorithm/ Working of the system: 
e Initialize the Raspberry Pi and all connected 
components. 
e Set up and configure the PCA9685 Servo Driver. 
3. Initialize and calibrate the servo motors. 
e Initialize the Raspberry Pi Camera for image 
capture. 
e Configure and set up the conveyor belt motor. 
Main Loop: 
e Continuously monitor the conveyor belt for the 
presence of an apple. 
e Capture an image of the apple when detected. 
e Process the captured image for ripeness 
detection. 
Image Processing: 
e Use image processing techniques to determine if 
the apple is ripe. 
e Define the sorting action based on ripeness. 
1. If ripe, proceed to pick and sort into the ripe 
box. 
2. If unripe, sort it into the unripe box. 
Robotic Arm Control: 
Control the robotic arm's servo motors for picking 
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and sorting. 
1. Adjust the arm to pick up the apple. 
2. Place it in the appropriate sorting bin based on 
ripeness. 
Conveyor Belt Control: 
e Control the conveyor belt motor to continue the 
process. 
e Repeat the loop for the next apple on the 
conveyor. 
Output Images: 


Figure 9 Unripe Apple 


4. Results 

Efficiency: Testing was performed with the help of 
10 specimen apples. Out of the total apples 6 were 
picked and 4 were rejected by the griper. When the 
10 apples were placed on the conveyor belt, the 
detection was done by the camera successfully which 
is riped or unriped simultaneously. The efficiency is 
decided of the model based on the acceptance of the 
gripper. So, we conclude the efficiency of the gripper 
is 60 percent. This percentage can be further 
increased to 80 percent by modifying the gripper 
specifications [6]. 


International Research Journal on Advanced Engineering Hub (IRJAEH) 


759 


IRJAEH 


Figure 10 Results 

Conclusion 

In this work, an automatic segregation system for 
fruit using color sensors has been successfully 
developed. The process of segregation is based on the 
ripeness color of the fruit. This project has been 
performed by quantitatively separating the ripe and 
unripe fruits using RGB range value. This process 
was repeated several times, and the results were 
almost identical. Based on the RGB data series, the 
ripe fruit showed a value exceeding, while the unripe 
fruit showed a value less than all tests. The difference 
between the two characteristics of the fruit is very 
significant with a difference of the value which can 
be categorized as under-ripe. This indicates that the 
probability of an error to occur in determining the 
characteristics of the fruit is slim and thus makes this 
system more reliable and robust. In the future, a set 
of sensors is proposed to detect a bunch of the other 
fruits as well. This further can speed up the 
segregation process and be more suitable for 
adaptation to the industry. Furthermore, the image 
processing technology is also great to introduce to the 
segregation system. 
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